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ABSTRACT 


A  new  discrete  reliability  growth  model  is  created  by 
modifying  the  often  used  Army  Material  Systems  Analysis 
Activity  (AMSAA)  continuous  reliability  growth  model.  The  new 
model  is  labeled  the  AMSAA-D  model.  Its  accuracy  is  evaluated 
and  compared  with  three  other  existing  discrete  reliability 
growth  models.  The  results  show  the  AMSAA-D  model  is  at  least 
as  accurate  as  the  other  models.  In  particular  it  is  more 
accurate  than  an  AMSAA  discrete  model  which  requires  computer 
supported  numerical  methods  to  calculate  the  reliability 
estimates  from  test  data.  The  AMSAA-D  reliability  estimates 
can  be  made  with  a  hand-held  calculator.  Computer  simulations 
were  used  to  generate  test  data  needed  for  the  evaluation.  The 
simulated  test  plan  assumes  that  repeated  tests  on  a  system 
are  performed  until  a  predetermined  number  of  failures  occur, 
at  which  time  a  design  change  is  made  to  the  system  to  improve 
its  reliability.  Ten  reliability  values  are  specified  to 
define  a  reliability  growth  pattern.  Five  hundred  replications 
of  each  growth  pattern  are  simulated.  For  each  replication, 
reliability  estimates  are  calculated  for  each  of  the  ten  sets 
of  generated  test  data  using  equations  from  each  of  the  four 
growth  models.  Averages  and  sample  mean  square  error  values 


IV 


across  the  500  replications  are  used  to  determine  accuracy. 
Sensitivity  of  the  AMSAA-D  model  to  the  number  of  failures 
before  system  modification  and  to  the  number  of  possible 
failure  causes  is  also  evaluated.  Results  of  all  evaluations 
are  presented  graphically. 


THESIS  DISCLAIMER 


The  reader  is  cautioned  that  computer  programs  developed 
in  this  research  may  not  have  been  exercised  for  all  cases  of 
interest.  While  every  effort  has  been  made,  within  the  time 
available,  to  ensure  that  the  programs  are  free  of 
computational  and  logic  errors,  they  cannot  be  considered 
validated.  Any  application  of  these  programs  without 
additional  verification  is  at  the  risk  of  the  user. 
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INTRODUCTION 


Early  in  the  acquisition  process  of  a  system,  a  variety 
of  test  programs  are  conducted  to  detect  weaknesses  in  its 
design  features  and  manufacturing  processes.  These  tests  are 
performed  under  tight  budget  constraints;  consequently  the 
number  of  component,  assembly  and  system  tests  is  severely 
limited.  These  development  testing  programs  are  designed  to 
induce  early  failures  and  determine  associated  failure  causes. 
This  information  is  needed  to  make  appropriate  changes  in  the 
design  and  production  processes.  Hopefully  these  changes  will 
yield  an  increase  in  the  reliability  of  the  system  under 
acquisition.  Current  DoD  policy  requires  reliability  growth 
management  and  assessment  programs  for  major  systems 
acquisitions. 

Reliability  growth  methodology  used  to  estimate  the  actual 
reliability  growth  patterns  should  be  able  to  track  a  variety 
of  growth  patterns  reasonably  well.  Many  reliability  growth 
models  have  been  developed  and  reported  in  the  literature. 
New  ones  are  continually  being  developed  and  proposed  for  use 
by  contractors. 

A  discrete  version  of  the  well  known  Army  Material  System 
Analysis  Activity  (  AMSAA  )  or  DUANE  model  is  analyzed  in  this 


thesis.  This  discrete  model  uses  the  number  of  observed  tests 
to  achieve  a  prescribed  number  of  failures  as  the  primary  data 
input.  This  model  is  evaluated  and  compared  with  other 
established  models  which  have  been  extensively  tested  and 
described  in  former  theses  [Ref.  1,2,3].  The  same  Monte  Carlo 
simulation  is  used  as  in  the  former  theses  to  make  the 
comparisons.  The  simulation  was  originally  developed  by  James 
E.  Drake  [Ref.  1],  then  twice  updated  and  enlarged  in  its 
performance  and  ability  to  track  different  reliability  growth 
patterns  by  James  D.  Chandler,  Jr.  [Ref.  2]  and  Pam  Markiewicz 
[Ref.  3].  The  program  and  its  features  are  well  described  in 
these  theses  and  will  not  be  included  here.  Only  the  program 
introduced  to  estimate  the  new  model  is  presented. 

In  Chapter  II  the  analysis  of  the  proposed  model  is 
developed,  the  simulation  data  presented  and  the  results 
evaluated.  A  summary  and  recommendations  for  further  studies 
is  presented  in  Chapter  III. 


II.  RELIABILITY  GROWTH  MODELING 


A  proposed  reliability  growth  model  should  be  able  to 
track  accurately  all  likely  reliability  growth  patterns. 
During  the  development  phase  of  a  system,  the  actual  pattern 
of  reliability  growth  is  dependent  on  many  factors. 
Reliability  progress  paths  may  be  steadily  increasing  or  they 
may  indicate  early  stagnation  and  degradation  prior  to 
continued  growth.  They  may  also  demonstrate  no  reliability 
progress  at  all  [Ref.  4].  Figure  1  shows  the  true  reliability 
growth  patterns  which  will  be  used  for  evaluation  purposes  in 
this  thesis.  The  different  patterns  are  numbered  from  1  to  6 . 
Pattern  numbers  cited  later  in  this  thesis  refer  to  Figure  1. 

Reliability  growth  usually  occurs  in  discrete  jumps; 
therefore  it  is  meaningful  to  break  the  system  tests  into 
subphases.  Tests  within  the  same  phase  have  the  same 
probability  of  success,  i.e.,  the  same  reliability. 
Reliability  may  change  from  phase  to  phase.  Hopefully  it  is 
increasing  as  the  items  under  test  undergo  change  from  phase 
to  phase. 


RELIABILITY  QROUTH  PATTERNS 

i.e 

— 1 

1 1-   T     I     I     I     1     .     1     •     1     1     1     ■     ' 

*    .    ^    -    -    -    -It    -    -    -    -0    - 

>    1    ,    . 

1 

- 

■                         P'' 

--''' 

- 

6  ;    / 
• ♦ — -,'■:» • -» • ■-- 

r-f<^-*' 

.— ..»  - 

• 

/■''b                       3           1    ' 'V 

/ 

■ 

•     /•      .      »      *  '-  X 

e.7B 

_... 

'l             y  ■      ^                                                          +  ^  .v^ 

;•  " y   -/^ N 1 ..J  .....^ 

" 

:  If                 ^  a  :       ^^i 

- 

^''!/    \  v^i 

3 

. 

■ '  *          *  ~  -^?^ 

•  '    .•■'  1                 j.y^                         \                —— 

pattam  1 

•H 

" 

•'/  .-                j< 

•Se.B 



U  /       \                      j--''''^-:  • -* 

pattarn  S 

■ 

"l!^ i" •^j;;:,:;-^^!  r      ; 

M 
-^ 

■ 

pattam  3 

f 

- 

1   ""■ 

pattam  4 

i                                    :        -x  - 

pattam  6 

e.25 

- 

\ ; i ♦• 

pattarn  6 

i 

e 



.  .  i  .  .  .  i  .  .  .  i  .  . 

.  i  1  • 

1  i 

0              2              4              6 

e 

!• 

tast  phasBB 

Figure  1  :  Possible  reliability  growth  patterns 


A.   THE  DISCRETE  AMSAA  MODEL 
1.   Theory 

The  Discrete  AMSAA  model  was  developed  in  1983  by  L. 
Crow  [Ref.  5]  It  stems  from  a  cumulative  continuous  model  of 
the  form 

K(t)  =  >t^ 
where  K(t)  denotes  total  number  of  failures  after  t  hours  of 
testing. 


In  the  discrete  model 

K.  =   2   M. 
j=l   J 
where  M.  denotes  the  number  of  failures  in  test  phase  j 

T.  =   I   N. 
3  =  1   J 
where  N.  denotes  the  number  of  tests  in  test  phase  j. 

The  model  then  states 


E(K^)  =  E(M^)  =  (1  -  R^)N^  =  \T^ 

E(K^)     =    E(M^  +  M2)  =  AT^+  (1  -  R2)N2  =  >T^ 


Thus   (1  -  R2)N2  =  >T^  -  AT^ 


Continuing  in  this  manner  the  AMSAA  discrete  model  states 


(1  -  R^)N^  =  AT^  -  XT^_^ 


Maximum  likelihood  estimates  for  X  and  P  are  used  to 
obtain  the  MLE  for  R..  The  estimates  are  updated  sequentially 
as  the  test  results  of  the  next  test  phase  become  available. 
2 .   Some  Results 

The  results  shown  in  the  following  figures  are  taken 
directly  from  a  thesis  by  Major  Rio  M.Thalieb  [Ref.  6]. 

Figure  2  shows  the  estimation  of  the  reliability 
growth  when  the  actual  reliability  is  following  pattern  1. 

Figures  16  and  17  in  Appendix  D  show  the  estimation 
of  the  discrete  AMSAA  model  for  patterns  2  and  4. 
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Figure  2  :  AMSAA  reliability  growth  estimation,  pattern  1 

It  is  clear  that  the  discrete  AMSAA  model  is  not  able 
to  track  all  possible  reliability  growth  patterns,  especially 
when  the  growth  is  not  steadily  concave  increasing.  This  is 
an  expected  result  due  to  the  fact  that  the  results  of  all 
phases  are  combined  to  estimate  the  reliability. 
B.   THE  AMSAA-D  MODEL 

The  model  analyzed  in  this  thesis  is  a  discrete  variation 
of  the  continuous  AMSAA  model  for  estimating  the  failure  rate 
of  a  system.  The  input  random  variables  to  the  continuous 
model   are  the  total   number  of   failures  and  the  total 


accumulated  test  time  up  to  the  point  of  estimation  [Ref.  4]. 
It  is  different  from  the  discrete  AMSAA  model  developed  by 
Duane  and  evaluated  in  the  thesis  of  R.M.  Thalieb. [Ref .  6]. 
The  equation  used  by  AMSAA  to  model  the  instantaneous  failure 
rate  f^^  is 


•TT 


=  b(l  -  a)TT~^  (  II.  1  ) 


where  TT  denotes  the  total  test  time  and  a  and  b  are 
parameters.  The  concept  for  this  model  was  first  presented  by 
Duane  [Ref.  7].  A  large  amount  of  development  test  data  he 
examined  indicated  that  the  logarithm  of  the  failure  rate  was 
a  linear  function  of  the  logarithm  of  the  total  accumulated 
test  time.  Comments  on  the  development  of  this  model  appear 
in  Appendix  E.  Woods  [Ref.  4]  showed  that  a  modification  to 
this  model,  using  regression  estimation  methods  rather  than 
maximum  likelihood  estimation  methods,  provided  a  more 
accurate  growth  model.  The  new  discrete  model  developed  here 
uses  similar  regression  estimation  methods. 
1.   Theoretical  analysis 

The  discrete  modification  of  the  continuous  AMSAA 
model  developed  here  should  be  distinguished  from  the  Discrete 
AMSAA  model  and  will  hereafter  be  referred  to  as  the  AMSAA-D 
model.  We  shall  adopt  the  following  notation  : 

j  or  k   :  test  phase  index,  corresponds  to  design  change 
index 


f-j   :  actual   failure  rate  during  phase  j  .   Equal  to 
probability  of  failure  for  a  test, 
f^j^    :  failure  rate  at  the  end  of  phase  k  as  determined  by 
the  model. 
Fj   :  total  number  of  failures  observed  in  phase  j 
N-;   :  number  of  trials  in  phase  j 

^tk  *  total   accumulated  number  of  trials  up  to  and 
including  phase  k 

(  Ntk  =  Ni  +  N2  +  ...  +  Nk  ) 
C-;   :  number  of  possible  failure  causes  in 
phase  j 
We  shall  model  the  failure  rate  f^]^  after  a  total 
number  of  tests  N^j^  have  been  accumulated  over  k  phases  by 


ftk  =  b(l  -  a)Ntk  (  II. 2  ) 


This  provides  a  discrete  version  of  the  model  given 
by  equation  (II. 1).  Regression  methods  similar  to  those  used 
by  Woods  [Ref.  4],  will  be  used  to  estimate  the  parameters  a 
and  b  chronologically  as  test  phases  are  completed.  That  is, 
all  observed  numbers  of  failures  F^,  F2 ,  •••  F-^  and  the 
observed  total  numbers  of  trials  N^,  N2 ,  ...  Njr  are  used  to 
obtain  the  current  estimate  of  f.,  ,  k=l,2  .... 

Each  test  phase  is  terminated  after  a  predetermined 
number  of  failures  has  occurred.  Consequently  the  number  of 
failures  per  phase  will  always  be  greater  than  0.  The  maximum 
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likelihood  estimate  for  f.  is  F./N. .  Its  expected  value  is 
larger  than  f.,  the  probability  of  failure  on  each  trial. 
There  are  numerous  ways  to  decrease  the  bias.  The  reduction 
factor  chosen  here  is  2C./(1+2C. )  primarily  because  it 
resembles  the  bias  reduction  factor  used  by  Woods  [Ref.  4]. 
Therefore  a  more  nearly  unbiased  estimate  Ij  for  f j  is  used 
where 


2Cj    Fj 

-^    -^     if  C^  >  3 


1  +  2Cj  Nj  1 

^j  =  {  (II. 3) 

0.85  -^  If  C.  =  1,2 
Nj  J 


A  rationale  for  equation  (II. 3)  can  be  developed  from 
a  continuous  analog.  If  X.  =  min(T.,t-),  i  =  1,2,  ...,N  where 
T.  ,   ...,  T„  are  iid  exponential  variables,  then  a  nearly 

A 

unbiased  estimate  X   for  the  common  failure  rate  \    is  given  by 

2N    F 
1+2N  1x7 


where  F  is  the  number  of  failures  (T.  <  t  ).  This  bias 
correction  factor,  2N/(1+2N)  is  developed  in  Reference  8. 
Woods  [Ref.  4]  uses  this  correction  factor  for  the  continuous 
model.  It  is  modified  here  by  replacing  N.  with  C . .  Table  I 
shows  a  comparison  of  both  correction  factors  2C ./ (1+2C . )  and 
2N./(1+2N.).  The  chosen  correction  factor  in  equation  (II. 3) 
is  a  constant  when  the  number  of  failure  causes  or  the  number 


of  subsystems  in  each  phase  are  fixed.  Table  I  shows  both  the 
smallest  and  the  largest  values  of  the  correction  factor 
2N./(1+2N.)  across  all  500  replications  and  six  growth 
patterns  for  j  =  1  and  j  =  10,  i.e.  for  phases  1  and  10.  These 
four  values  are  grouped  by  number  of  failure  causes.  The 
number  of  failure  causes  is  an  input  parameter  for  the 
simulation  program.  The  number  of  failure  causes  is  important 
when  a  fraction  of  a  failure  for  a  specific  cause  can  be 
removed  after  a  fixed  number  of  successfull  system  tests 
occur  without  failure  for  the  same  cause.  This  failure 
reduction  process  is  called  failure  discounting  and  is 
permitted  by  specifying  input  parameters  to  the  computer 
program  used  to  run  these  simulations.  Failure  discounting  was 
not  allowed  in  any  simulation  in  this  thesis. 

TABLE  I  :  COMPARISON  OF  CORRECTION  FACTORS 


#  of  failure  causes  C_ 

1 

3 

5 

7 

AMSAA-D   20^/(1+20^) 

0.85 

0.8571 

0.9091 

0.9333 

continuous 

AMSAA 

model 
2Nj/(l+2Nj) 

min 
phase  1 

max 

0.8571 
0.9524 

0.8571 
0.9565 

0.8571 
0.9565 

0.8571 
0.9333 

min 
phase  10 

max 

0.9524 
0.9915 

0.9091 
0.996 

0.96 
0.9948 

0.9231 
0.9922 

Least  square  estimation  procedures  are  used  to  obtain 
estimates  for  a  and  b  as  the  testing  in  each  new  phase  is 
completed.  This  provides  update  estimates  for  a  and  b  and  thus 
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for  the  failure  rate  f  x.  •  ,  j  =  1,2  ....  Equations  for  the  least 
square  estimates  were  first  developed  by  Neal  [Ref.  9].  He 
uses  the  transformations 

Xj  =  In  N^j     and 

Yj  =  m  ftj 
to  obtain  a  linear  relationship  between  Y .   and  X .  .  His 
equations  for  the  least  squares  estimates  a.-^    and  fej,  are  as 
follows  : 


k         _   k 

2   X.Y.  -  Y,   2   X. 
j=l   3  3     k  j=i  3 

a,  =  —^ ^ 

k     _    k        k    2 

X,   2   X.  -   2   XT 

k  j=i  J        j=i  J 


where 


^k 

= 

1 

e^k  +  ^A 

1    - 

A 

^k 

1 

k 

2 

^j 

Yk 

1 
k 

2 

^j 

k  >  2. 

The  regression  estimation  can  be  made  only  after  two 
phases  of  testing  have  been  completed.  Therefore  the  failure 
rate  is  estimated  for  k  >  2  as 

A 

^tk  =  (  1  -  ^k  )  ^k  N~^^  (  II.4  ) 

The  term  f.   =  f   is  estimated  with  equation  (  II. 3  ). 
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2.   Reliability  estimate  as  a  linear  function  of  the 
failure  rate  estimate. 

To  compare  the  results  of  this  model  with  other 
models,  it  is  necessary  to  convert  the  failure  rate  estimate 
into  a  reliability  value. 

A  discrete  failure  rate  is  defined  as 


_  P[N=n]   „  _  .  .  - 


where  the  random  variable  N  is  the  number  of  cycles  (trials) 
up  to  and  including  the  cycle  on  which  the  r  failure  occurs 
[Ref.  10].  For  the  geometric  distribution  r  =  1  and  f  =  1-p. 
When  estimating  system  reliability  in  each  phase,  one  basic 
assumption  is  that  the  unknown  reliability,  p,  stays  constant 
within  that  phase.  Even  though  we  are  observing  the  number  of 
trials  to  the  r  failure  with  common  probability  of  success 
p  on  each  trial,  we  shall  take  the  failure  rate  to  be  the 
unreliability  1-p.  This  means  that  the  failure  rate  is  just 
the  probability  of  failure  on  a  single  trial  inside  one  single 
phase.  Consequently  one  method  for  converting  the  least 
squares  failure  rate  estimates  into  reliability  estimates  is 
from  the  expression  : 

where  f.,   is  given  in  equation  (II. 4).  In  order  to  avoid 
subscripts  in  graphs,  equation  (II. 6)  will  be  written  as 
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R(N)  =  1  -  F(N)  in  the  graphs.  In  the  text  that  follows  it  is 
referred  to  as  "the  linear  model".  A  more  detailed  discussion 
of  the  relationship  between  failure  rate  and  reliability  for 
discrete  distributions  is  provided  in  Appendix  C. 

3.  Reliability  estimate  as  an  exponential  function  of  the 
failure  rate  estimate. 

Analysis  of  the  first  simulations  indicated  that  the 
linear  model  consistently  underestimated  the  actual  system 
reliability  on  the  average.  To  increase  the  value  of  the 
average  point  estimate,  another  equation  was  introduced  for 
estimating  reliability  using  the  failure  rate.  This  equation 
is 

^tk  =  e"  tk  (  II.7  ) 

and  is  referred  to  as  "  the  exponential  power  model  "  in  the 
text.  In  the  figures  it  is  noted  as 

R(N)  =  exp(f (N) ) . 

4.  Simulation  results 

The  simulations  were  replicated  500  times  for  each 
combination  of  input  parameters.  These  parameters  are 

-  the  phase  reliabilities  p^  . . .  p^^.  Six  different  sets  of 
these  10  values  establish  six  different  reliability  growth 
patterns, 

-  the  number  of  failure  causes  per  phase 
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-  the  number  of  failures  before  corrective  action  in  each 
phase. 

Table  II  gives  an  overview  of  all  simulated  parameter 
combinations . 

Table  II  :  ALL  SIMULATED  PARAMETER  COMBINATIONS 


#  of 
causes 

#  Of 
fail/phase 

pattern 

1 

2 

3 

4 

5 

6 

3 

3 

X 

X 

X 

X 

X 

X 

1 

X 

X 

X 

X 

X 

X 

2 

X 

X 

X 

X 

X 

X 

5 

3 

X 

X 

X 

X 

X 

X 

5 

X 

X 

X 

X 

X 

X 

7 

X 

X 

X 

X 

X 

X 

7 

3 

X 

X 

X 

X 

X 

X 

During  each  replication  the  estimated  system 
reliability  as  well  as  the  estimation  error,  Err.,  was 
computed.  Err.  is  defined  as 


Err .  =  R.  -  R.  • 
:     D     t:) 


(  II. 8  ) 


where 


''j 


actual  system  reliability  in  phase  j 

R.  .  :  estimated  system  reliability  at  the  end  of 

phase  j 

These  computations  are  performed  for  both  AMSAA-D 

models  (  the  linear  and  the  exponential  power  model  ) .  They 

are  also  computed  for  the  Maximum  Likelihood  Estimation  model 

and  the  exponential  Regression  Estimate  model;  hereafter 
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referred  to  as  the  MLEwd  and  Exp. Reg. Estimate  models.  These 
models  have  been  evaluated  by  Chandler  [Ref.  2].  Additional 
comments  on  these  models  are  given  in  Appendix  E.  After  all 
replications  are  completed  the  average  system  reliability 
estimate  and  the  Mean  Square  Error  are  computed  for  each 
estimation  method  as 


1   nrep 


R^.=  — —     2    (RJx.^  for  all  j  (  II. 9a  ) 

t^    nrep  .^      ^  I'tj  -"  ^ 

nrep      ^ 

MSE.  =  — —      1      [Err.]"^  for  all  j  (  II .  9b  ) 

3     nrep  ._^  •■    i-'  -"  ^       ' 


where  j      :  phase  index 

nrep   :  total  number  of  replications  (  =  500  ) 

"a 

R .  .    :  average  reliability  estimate  over  nrep 

replications  for  phase  j 
MSE    :  Mean  Square  Error 

This  average  reliability  estimate  and  the  Mean 
Square  Error  are  plotted  and  graphed  to  show  the  performance 
of  the  growth  models. 

The  MLEwd  and  the  exp. Reg. Estimate  models  are  used  to 
compare  the  performance  of  both  AMSAA-D  models  relative  to 
these  former  established  growth  models.  The  AMSAA-D  model  does 
not  allow  the  application  of  any  discounting  method.  Therefore 
the  comparisons  are  all  made  without  any  discounting  applied 
to  the  MLEwd  and  the  exp. reg. Estimate. 
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a.      Sensitivity    of    the    reliability    growth    estimation    with 
respect  to  different  growth  patterns 
The  evaluation  of  the  sensitivity  of  both 
AMSAA-D  growth  models  with  respect  to  different  growth 
patterns  is  discussed  with  the  following  parameter  setting  : 

-  number  of  failure  causes  :    5 

-  number  of  failures/phase  :    3 

The  results  and  conclusions,  however,  are  in  general  the  same 
for  all  other  parameter  combinations  as  is  shown  in  Figures 
18  -  35  in  Appendix  D. 

Figure  3  illustrates  the  performance  of  both 
AMSAA-D  models  (  linear  and  exponential  )  when  the  true 
reliability  growth  follows  the  path  stipulated  in  pattern  1. 

The  exponential  power  model  in  the  first  6  phases 
far  overestimates  the  actual  reliability.  The  linear  model 
tracks  the  convex  growth  pattern  very  well  over  all  phases. 
Also  it  underestimates  the  actual  reliability,  which  is 
usually  a  desired  behavior.  In  comparison  with  the  MLEwd  and 
the  exp. Reg. Estimate,  the  linear  model  performs  better  over 
all  phases  with  respect  to  the  average  of  the  point  estimates 

A  A 

\l'  ■^t2'  •  •  • 

The  Mean  Square  Error  for  all  four  methods  is 

shown  in  Figure  3.  The  exponential  power  model  is  the  worst 

estimation  model  with  respect  to  the  average  reliability 

estimate,  but  it  has  the  smallest  Mean  Square  Error  and  is 

less  dependent  on  the  phase.  But  from  phase  8  on,  all  models 

have  nearly  the  same  Mean  Square  Error. 
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Figure  3  :  Reliability  growth  estimation,  pattern  1 
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Figure  4  illustrates  the  performance  of  all  models 
under  pattern  2 . 

Both  AMSAA-D  models  track  the  decline  of  the 
actual  reliability  very  well.  The  linear  model  again  always 
underestimates  the  actual  system  reliability,  and  the 
exponential  power  model  overestimates  the  actual  system 
reliability  up  to  phase  6.  It  also  smoothes  the  decline  and 
the  following  incline  of  the  actual  reliability. 

The  linear  model  performs  better  with  respect  to 
the  average  reliability  estimate,  when  compared  with  the  MLEwd 
and  the  exp. Reg. Estimate .  Also,  from  phase  3  on,  both  AMSAA- 
D  models  have  the  smaller  Mean  Square  Error  and  this  Mean 
Square  Error  is  not  very  strongly  influenced  by  the  phase  as 
is  the  case  for  the  MLEwd  and  the  exp. Reg. Estimate. 

For  pattern  3  the  general  behavior  of  all  models, 
as  shown  in  Figure  5,  is  similar  to  the  behavior  for  patterns 
1  and  2 . 

All  models  perform  well  under  pattern  4  and  5  as 
it  can  be  seen  in  Figures  6  and  7.  For  pattern  5  only  the 
exp. reg. Estimate  shows  an  overestimation  of  the  actual 
reliability  over  all  phases.  Again,  the  decline  of  the  Mean 
Square  Error  from  phase  1  to  phase  3  for  all  models  is  very 
significant. 

Figure  8  shows  the  behavior  under  pattern  6.  All 
models  underestimate  the  actual  reliability  quite  well  in  the 
average.  The  Mean  Square  Error  is  nearly  the  same  for  all 
models  and  is  also  nearly  constant  over  all  phases.  Only  the 
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Figure  4  :  Reliability  growth  estimation,  pattern  2 


19 


ReLXABZLXTY    OROWTH    BSTZHATXON 

P«tt«m  3,    BMu—   ■   6,    railura/'prMwa   ■   3 

•  .  93 

1 

' 

1        j        1        1     '  1        j        I        1  '     1        1        1  ■     1        1        1        I        1        1        1 

1 

i 1 1 ^ ........^^..............._^. - 

•  .83 

— i 

i                 i ""'"    "p^^-" 

■     I 

■     i 

j          ./,--'---! ♦ *''                                                 ;    " 

•  .73 

— ■! 

i ,il^.r.-..r;.r.^:---;-:-«.-r;.-.--.*; j.J 

3 

j 

i   ////                                                                                                        \    '- 

«J 

1 

■•* 

: 

'i        /  fl                                                                                                                                                                                  i       - 

-< 

1 

f    ///                                                                                                                !     . 

B   •.63 

-  -I  

i 

.•■      \/fi'                                                                                                             i    - 

~i 

/  '  /                                                                                                               ' 

•H 
t 

e.63 

j 

A*';                                                                     — .—     actual  iral.                        ; 

-     1 

>' 

/  i 

/^/                                                                           -  +  •     R<N)«l-ir<N> 

/^/ ! !•  — 

-     ! 

/'.■  ';                                                                          ..»       R(N)»»><p(f  <N)> 

■    i 

A''/    j                                                                          ..■..     HUE   with  dlaeountlnq 

; 

J 

•'.'    i                                                                                    !   " 

•  .43 

-  ■■! 

i 

•,' .' i vijt-      axp.  R»tj.e»tlm«ta          j  — 

/              j                                                                                                 1                                                ;                                                ':       •■ 

-      1 

*  » 

it 

i                                                                                                 !                                                :                                                :       - 

1 

•  .33 

a 

:                                                1                                                ;                                                :                                                : 

J       ;       ,       .       i       .       .       .       i       .       .       .       1       .       .       .       i       .       .       1       i 

• 

S                              4                               e                               8                              !• 

p^MMl■■ 

MEAN    SQUARE    ERROR 

Pat tarn   3,    eauaaa    ■    B,    fallura/pHasa    >    3 

•  .•e 

! 

....: 1 : : ; ; : ; j.....;.,  .h(NUi-!r(h^>    ■ : jz. 

1                                                 j    ....     n(N)E»^(f <N>)            j    ~_ 

K, 

j                                                                     i      -  ■  .     HLE    with   discounting     ~ 

•  ••6 

0  - 

'-'T  -■*                                                                                                                                         i     - 

n 

""^^x 1 '.;»; wxpi  Rag;  Eatlmata          ! 

•  ••4 

-  i 

♦  - 
a 

—  ♦  \  ^                                  1                                        i  - 

! 

'     \  \      '^                                                                      :                                                                                          :      . 

$i    •.•S 

-      1 

"\  ^                   ■.         i                                       1  - 

V ^V 'V ■■■< >.^ i f- 

z 

-    ; 

■    i 

! 

•  .OS 

j, 



\. X >.,.   ..^ "..>,4 ;.v =  .._ 

: 

j 

»..                      \            1                                       i       V         n                                                          i      - 

\    1                             "v.    ^        i                 .--?  - 

•  .•1 

'-• j 

■•,.     \  i       • V\    i        --'    ,-4  - 

'~y:—z:::::^-r::^:-.i^^^^^^^^ |- 

i                  ?■-..               t      • f  - 

• 

~"i 

1 

1 j ::::::t:":""*::-:.:.:-:.i_: 

i       .       .       .       i       .       .       .       i       .       .       .       i 

• 

a                                4                                6                                8                               !• 

phaaaa 

Figure  5  :  Reliability  growth  estimation,  pattern  3 
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Figure  6  :  Reliability  growth  estimation,  pattern  4 


21 


e.e3 


3 


•  .67 


».6e 


•  .33 


•  .•8 


•.•e 


•  .•4 


•  .0S 


RELIABILITY  QROVrTH  ESTIMATION 
Pattam  B^  oauaas  ■  B.  Tallura/'pHji**  ■  3 


"T" 


"T 


..  jr- 
i 


!S5i.:.rH5:::::ri::.::Srr:-ir.i::  ,5 


I 

aotual  iral. 
R<N)"l-|r<N) 
R(N>3a><p<r<N)> 
ri-E   with   diacoun 
Rxp .  Rat) .  Eat  imat 


ting 


!• 


pHaaaa 


MEAN  SqUAPE  ERROR 
Pattam  6,  oauaaa  ■  8,  f aiiura/pHaaa  *  3 


I 

•   11)1 

■ 

' 

■-T 

' 

! 

-  4- 

h<N: 

-iJr<»V) : : }..._ 

"a)<p<f(N))      ; 
with  diacounting 
Rag.Eatiaata 

i 
j 
1 

1        i 

1 

i 

— ■  ■ 
■-W- 

R<N) 

HLE 

axp. 

1 

\      1 

\\     \ 

V  \              j 

i  ■■ 

I                 : 
1                 ; 

I                 j 

i                 i 

*   ^  1 
"/^   V  i 

•.  '^    >  i 

\ t\..:^- 

i  . 
i  - 

1 

i 

:   K 

i " 

1 

*'*^'.. 

1 

^■^•4-*''' 

- 

•  - 

4-' 

'  - 

.a 

—  ■♦■  —  w 

-f^ - 1  - 

i 

i 

, 

, 

i 

» 

, 

i 

" 

1   1 

i      ,      .      .      i 

!• 


phaaaa 


Figure    7    :    Reliability   growth   estimation,    pattern    5 
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Figure  8  :  Reliability  growth  estimation,  pattern  6 
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exp. Reg. Estimate  shows  a  slight  decline  in  the  Mean  Square 
Error  from  phase  1  to  10. 

Figures  9  and  10  illustrate  the  behavior  of  both 
AMSAA-D  models  with  respect  to  the  Mean  Square  Error  and  with 
phases  and  patterns  as  variables. 


Figure  9  :  Linear  model,  Mean  Square  Error  as  a  function  of 
phases  and  patterns 

Both  models  show  a  high  Mean  Square  Error  in  the 
early  phases.  This  is  reasonable  because  in  the  early  phases 
less  information  is  available  to  get  accurate  reliability 
estimates.  The  exponential  power  model  (  Figure  10  )  shows  a 
constant  decline  of  the  Mean  Square  Error  for  pattern  1, 
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Figure  10  :  Exponential  power  model,  Mean  Square  Error  as  a 
function  of  phases  and  patterns 

whereas  for  the  pattern  2-5  the  Mean  Square  error  drops 
rapidly  after  phase  1  and  stays  nearly  constant  thereafter. 

For  both  models  pattern  6  is  less  dramatic  because 
both  models  show  a  nearly  constant  low  Mean  Square  Error  over 
all  phases. 

b.       Sensitivity   of    the    reliability   growth    estimation    to    the 

number  of  failure  causes. 

To  evaluate  the  sensitivity  of  the  reliability 
growth  estimation  to  the  number  of  failure  causes  the 
simulation  was  run  for  each  pattern  with  the  number  of  failure 
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causes  set  at  3 ,  5  and  7.  Figure  11  and  the  Figures  36  to  40 
in  Appendix  D  show  three  dimensional  plots  of  the  Mean  Square 
Error  as  a  function  of  the  phase  and  the  number  of  failure 
causes  for  the  linear  and  the  exponential  power  model.  The 
Mean  Square  Error  was  chosen  as  a  measure  of  the  sensitivity, 
because  the  estimation  of  the  actual  reliability  doesn't  show 
great  differences  due  to  the  change  in  the  number  of  failure 
causes.  The  graphs  of  the  reliability  estimation  and  their 
related  Mean  Square  Error  for  all  different  parameter  settings 
is  enclosed  in  Appendix  D  (  Figures  18  -  35  ) . 

As  can  be  seen,  the  general  behavior  of  the  growth 
estimation  due  to  the  number  of  failure  causes  is  relatively 
independent  of  the  simulated  growth  pattern.  Also  the 
difference  between  the  two  models  is  not  very  large.  Both 
models  show,  in  general,  an  increasing  Mean  Square  Error  when 
the  number  of  failure  causes  is  increased.  This  increase  is 
overall  a  little  steeper  for  the  linear  model,  but  it  becomes 
less  dependent  in  the  number  of  causes  in  later  phases. 

The  exponential  power  model  shows  a  more 
differentiated  behavior.  It  has  a  steep  increase  of  the  Mean 
Square  Error  for  pattern  1  in  the  first  5  phases  as  the  number 
of  causes  is  increased.  This  is  true  in  all  other  patterns 
only  for  phase  1.  For  all  other  patterns  and  phases  greater 
than  1  the  Mean  Square  Error  of  the  exponential  power  model 
is  only  slightly  increasing  with  increasing  number  of  causes. 
The  slope  is  reduced  even  more  within  the  last  phases. 
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Figure  11:  Mean  Square  Error  as  a  function  of  phases  and 
failure  causes,  both  AMSAA~D  models,  pattern  1 
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To  catch  the  general  trend  in  a  different  way  a 
total  average  Mean  Square  Error  was  computed  as 

-   npat     .    nphase 
tot.avg.MSE  =    Hpit  p!^  (  T^pY^    ■!,    "^^jp  >    <"•"' 

where 

jp    :  Mean  Square  Error  as  in  equation  (II. 9), 
j        :  phase  index  (  here  1  to  10  ) , 
p        :  pattern  index  (  here  1  to  6  ) , 
nphase   :  number  of  phases  (  here  10  )  and 
npat     :  number  of  patterns  (  here  6  ) . 

The  result  is  shown  in  Figure  12.  It  also  affirms 
the  trend  estimation  from  the  three  dimensional  graphs.  The 
total  average  Mean  Square  Error  increases  for  both  models. 
This  is  also  true  for  the  other  two  models,  MLEwd  and 
exp.Regr. Estimate,  shown  for  comparison.  But  the  slope  of  the 
linear  model  is  steeper  than  the  slope  of  the  exponential 
power  model  when  increasing  the  number  of  causes  from  3  to  5 . 
c.  Sensitivity  of  the  reliability  growth  estimation  to  the 
number  of  failures  per  phase 

The  evaluation  of  the  sensitivity  of  the 
reliability  growth  estimation  to  the  number  of  failures  per 
phase  was  done  with  the  following  parameter  settings  : 

-  all  patterns  (  1  to  6  ) 

-  number  of  failure  causes  =  5 
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Figure  12  :  Average  Mean  Square  Error  as  a  function  of  failure 
causes 

Figure  13  and  Figures  41  to  45  in  Appendix  D  show 
the  three  dimensional  plots  of  the  variation  of  the  Mean 
Square  Error  over  all  phases  with  varied  failures/phase  for 
all  patterns. 

From  all  Figures,  it  can  clearly  be  seen  that  the 
greatest  impact  on  the  reduction  of  the  Mean  Square  Error 
occurs  where  the  number  of  failures/phase  changes  from  1  to 
2.  In  general  an  increase  in  the  number  of  failures/phase  from 
1  to  2  will  reduce  the  mean  Square  Error  by  at  least  50%  in 
most  of  the  cases.  This  is  relatively  independent  of  the 
phase.  But  the  reduction  in  phase  1  is  always  the  smallest  for 
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Figure  13  :  Mean  Square  Error  as  a  function  of  phases  and 
failures  per  phase,  both  AMSAA-D  models,  pattern  1 
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each  pattern.  Note  that  there  is  no  decrease  of  the  Mean 
Square  Error  for  the  exponential  power  model  in  the  first 
phase  for  the  patterns  1  to  5  when  the  number  of 
failures/phase  is  increased.  Only  pattern  6  shows  a  decrease 
in  phase  1  also. 

The  variation  of  the  total  average  Mean  Square 
Error  over  all  phases  and  patterns,  computed  as  in  equation 
11.10,  with  increasing  number  of  failures/phase  is  shown  in 
Figure  14.  It  again  affirms  in  general  the  trend  evaluated 
from  the  three  dimensional  graphs.  The  greatest  reduction  of 
the  Mean  Square  error  is  achieved  by  increasing  the  number  of 
failures/phase  from  1  to  2 .  This  results  in  an  average 
decrease  of  the  Mean  Square  Error  from  0.08  to  0.03  for  the 
linear  model  and  from  0.03  to  about  0.015  for  the  exponential 
power  model.  This  result  is  true  in  general  for  the  Maximum 
Likelihood  Estimation  and  the  exponential  Regression  Estimate, 
which  are  also  shown  in  Figures  13  and  14  for  comparison. 
d.       Comparison  with  the  discrete  AMSAA  model 

The  results  show  the  AMSAA-D  model  is  more 
accurate  than  the  discrete  AMSAA  model.  The  AMSAA-D  model 
appears  to  account  more  accurately  for  different  reliabilities 
in  different  phases,  but  still  uses  information  from  previous 
phases  to  yield  more  accurate  estimates  of  reliability  at 
later  stages  in  the  growth  process.  It  makes  good  use  of  the 
fact  that  the  actual  reliability  is  not  constant  over  all 
phases,  but  stays  constant  within  a  phase.  The  discrete  AMSAA 
model  is  not  able  to  track  all  different  possible  patterns  of 
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Figure  14   :  Average  Mean  Square  Error  as  a  function  of 
failures  per  phase 

reliability  growth  as  can  be  seen  in  Figure  15,  which  is  taken 
from  R.Thalieb  [Ref.  6]  for  pattern  2  with  3  failures/phase 
and  1  failure  cause. 
e.  Conclusion 

The  linear  AMSAA-D  model  performs  reasonably  well 
and  is  relatively  independent  of  the  pattern  of  actual 
reliability  growth.  It  always  underestimates  the  actual 
reliability  and  is  therefore  a  conservative  approach.  It  also 
performs  at  least  as  good  as  the  formerly  established  MLEwd 
and  exp.Regr .Estimate  models  when  using  Mean  Square  Error  as 
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Figure  15  :  Comparison  of  the  discrete  AMSAA  model  with  the 
AMSAA-D  model,  pattern  2 

a  measure  of  performance.  It  is  not  very  sensitive  to  the 
number  of  failure  causes  and  this  low  sensitivity  is  even 
decreasing  within  later  phases  of  testing.  When  the  number  of 
failures/phase  are  increased  from  1  to  2 ,  the  Mean  Square 
Error  can  be  reduced  by  about  50%. 

The  exponential  power  model  has  a  very  low  Mean 
Square  Error.  In  general,  it  is  about  50%  less  than  the  Mean 
Square  Error  of  the  other  models.  It  overestimates  the  actual 
reliability  in  the  early  phases  for  nearly  all  of  the  six 
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growth  patterns.  The  impact  on  accuracy  due  to  an  increase  in 
the  number  of  failure  causes  is  nearly  the  same  as  for  the 
linear  model.  The  same  can  be  said  for  the  impact  of  an 
increase  in  the  number  of  failures/phase. 

The  linear  AMSAA-D  model  is  a  conservative,  but 
reasonable  accurate  tool,  for  estimating  actual  system 
reliability  growth.  It  is  not  very  sensitive  to  the  number  of 
failure  causes  and  an  increase  in  the  number  of  failures  per 
phase  can  reduce  the  Mean  Square  Error  significantly. 
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III.  SUMMARY,  RECOMMENDATIONS 


A.   SUMMARY 

The  objective  of  this  thesis  was  to  evaluate  the  accuracy 
of  a  discrete  modification  of  the  continuous  AMSAA  model 
(  AMSAA-D  model  )  for  the  reliability  growth  estimation.  The 
sensitivity  of  the  model  to  different  growth  patterns  and 
system  parameters  as  the  number  of  subsystems,  the  number  of 
phases  and  the  number  of  failures  till  phase  termination  was 
evaluated.  The  accuracy  of  this  model  was  compared  with  that 
for  the  MLEwd  and  the  exponential  regression  estimate  models 
[Ref.  1,2,3].  Although  possible  improvements  due  to  different 
discounting  and  weighing  methods  for  the  MLEWD  and  the 
exponential  regression  estimate  models  have  been  demonstrated 
in  these  former  theses,  comparisons  in  this  thesis  were  made 
without  using  discounting  or  weighting  of  data. 

Throughout  the  simulations  the  AMSAA-D  model  illustrated 
a  reasonable  accuracy  and  was  at  least  as  good  as  the  MLEWD 
or  the  exponential  regression  estimate.  It  is  able  to  track 
all  possible  growth  patterns  reasonably  well.  Its  sensitivity 
to  the  number  of  failure  causes  and  the  number  of  failures 
within  a  phase  compares  favorably  with  that  of  other 
established  models. 
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B.   RECOMMENDATIONS 

The  following  are  recommendations  for  further  study  : 

1.  The  AMSAA-D  model  should  be  modified  by  using  the 
minimum  variance  unbiased  estimate  of  the  failure  rate 
instead  of  the  failure  rate  estimate  using  a 
correction  factor. 

2.  Since  the  exponential  regression  estimate  can  be 
improved  by  weighting  the  data,  similar  methods  should 
be  analyzed  for  the  AMSAA-D  model. 
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APPENDIX  A 
USER'S  GUIDE  TO  :  RELIABILITY  GROWTH  WITH  THE  AMSAA-D  MODEL 

(  DISCONT  ) 

1.  Introduction 

2.  The  DISCONT  EXEC  Al  file 

3.  The  input  data  file 

4.  The  output  file  GROWTH  EST  Al 
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APPENDIX  A  :  USER'S  GUIDE  TO 

RELIABILITY  GROWTH 

WITH  THE 

AMSAA-D  MODEL  (DISCONT) 


1 .  Introduction 

In  order  to  use  the  Fortran  program  DISCONT  the  user 
must  possess  three  files  : 

-  Input  file  "like"  PT1C5F3  DATA  Al 

-  DISCONT  FORTRAN  Al 

-  DISCONT  EXEC  Al 

A  sample  of  each  of  these  files  along  with  a  sample 
run  using  DISCONT  EXEC  Al  and  a  sample  output  is  contained  in 
Appendix  B.  The  input  file  and  the  exec  file  can  be  changed 
to  meet  the  needs  of  the  user.  In  the  current  form,  the  exec 
file  produces  a  large  number  of  intermediate  calculations  for 
both  the  DISCONT  FORTRAN  program  and  Lt  P.  Markiewicz's 
program  DRG  FORTRAN  [Ref.  3],  These  calculations  may  not  be 
of  interest  and  therefore  may  be  eliminated  with  no  effect  to 
the  simulation.  A  detailed  explanation  of  each  file,  which  is 
created  in  addition  to  Lt  P.Markiewicz ' s  version  is  contained 
in  the  following  sections. 

2.  The  DISCONT  EXEC  Al  file 

The  DISCONT  EXEC  Al  contains  all  necessary  file 
definitions  and  commands  needed  to  run  DISCONT  FORTRAN  Al. 
Appendix  B  2.  shows  a  sample  run  using  DISCONT  EXEC  Al. 
DISCONT  FORTRAN  Al  is  an  extension  of  the  DRG  FORTRAN  Al 
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program  of  Lt  P.Markiewicz  [Ref.  3].  Therefore  in  addition  to 
the  output  created  for  the  discretized  continuous  models  there 
are  all  outputs  and  models  available,  which  have  been  used  by 
Lt  P.Markiewicz   [Ref.  3]. 

For  the  use  of  DISCONT  FORTRAN  Al  it  is  necessary  to 
specify  an  Input  file  in  line  24  of  DISCONT  EXEC  Al.  This 
input  file  must  have  all  necessary  inputs  in  the  same  format 
as  mentioned  in  Appendix  A  of  Lt  P.Markiewicz  thesis  [Ref.  3]. 
The  filename  and  type  can  be  chosen  free  as  long  as  it  matches 
the  format  of  the  CP/CMS  requirements  of  a  maximum  of  eight 
characters. 

For  the  purpose  of  this  thesis,  42  different  input 
files  have  been  created  to  allow  the  evaluation  of  the 
sensitivity  of  the  model  to  the  following  different  input 
parameters 

-  type  of  growth  pattern 

-  number  of  failure  causes  and 

-  number  of  failures  per  phase. 

The  filenames,  used  in  the  42  input  files,  express  the 
setting  of  these  different  input  parameters.  For  example,  the 
filename  PT1C5F3  means 

-  pattern  :  i 

-  number  of  failure  causes  :  5 

-  number  of  failures/phase  :  3 

The  full  name  of  the  input  file  is  then 
PT1C5F3  DATA  Al . 
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The  output  for  the  AMSAA-D  models  as  well  as  the 
results  of  the  models  chosen  for  comparison  is  written  to 
GROWTH  EST  Al .  A  sample  output  is  given  in  Appendix  B. 

Also  a  control  output  is  available.  It  allows  us  to 
track  the  intermediate  results  of  single  simulation  runs  for 
the  first  and  last  10  replications,  independent  of  the  number 
of  replications  chosen.  This  control  output  is  written  to 
GROWTH  CONTROL  Al . 

The  next  output  file  is  FAILURE  RATE  Al.  It  contains 
the  actual  reliability  as  well  as  the  average  of  the  estimated 
failure  rates  of  each  replication,  which  is  the  basis  for  the 
computations  of  the  estimated  reliability  in  the  linear  and 
the  exponential  power  model. 

The  last  output  file  is  MSE  COMPARE  Al.  This  file 
contains  results  used  to  evaluate  the  influence  of  changes  in 
the  number  of  failure  causes  and  the  number  of  failures/phase. 
It  contains  the  average  Mean  Square  Error,  averaged  over  all 
phases  from  the  distinct  Mean  Square  Error  for  each  phase, 
computed  for  the  four  growth  models 

-  1  ....  linear  model 

-  2  ....  exponential  power  model 

-  3  ....  MLE  with  discounting 

-  4  ....  exponential  Regression  Estimate 

In  addition  to  these  files,  related  to  the  AMSAA-D 
models,  all  results  of  Lt  P.Markiewicz ' s  simulation  program 
DRG  FORTRAN  Al  are  computed  and  written  to  the  same  output 

40 


files  as  in  their  original  program.  The  only  difference  is 
that  the  filenames  have  in  addition  an  "X"  in  front  of  their 
original  name.  In  this  way  it  is  possible  to  run  both 
simulations,  DISCONT  and  DRG,  one  after  the  other  without 
overwriting  the  first  created  output  files.  Therefore  the  file 
descriptions  in  Appendix  A  of  the  thesis  of  Lt  P.Markiewicz 
are  totally  valid. 

3 .   The  input  data  file 

These  instructions  should  be  used  in  connection  with 
a  sample  input  (  see  Appendix  B  2.).  The  sequence  of  all  data 
inputs  is  mandatory  as  it  is  shown  in  the  sample  input  file. 
The  input  is  read  from  file  device  number  10. 

For  the  discretized  continuous  models  the  input  can 
be  the  same  as  for  the  DRG  FORTRAN  program  of  Lt. 
P.Markiewicz.  All  steps  noted  in  her  thesis  in  Appendix  A  3. 
are  still  valid.  The  only  difference  is  that  the  first  line 
of  the  input  file  should  contain  the  filename  of  the  input 
file  to  get  this  remark  stated  on  top  of  the  output  file 
GROWTH  EST  Al .  Due  to  this  additional  first  line  the  only 
change  in  the  steps  of  Appendix  A  3  of  Ref.  3  is  that  the 
mentioned  line  numbers  must  be  increased  by  one.  But  all  other 
input  variations/possibilities  stay  the  same. 
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4.   Output  file  GROWTH  EST  Al 

The  output  file  shows  in  its  first  line  the  filename 
of  the  input  file  used  for  the  actual  simulation  run.  The 
names  of  the  variables  are  : 


-  PHASE 

-  ACTREL 

-  LINREL 

-  MSEL 

-  RELEXP 

-  MSEE 

-  MLEWD 

-  MSEWD 

-  REGEST 


self  explanatory 

actual  reliability 

reliability  estimate,  using  the  linear  model 

Mean  Square  Error  of  the  linear  model 

reliability  estimate  using  the  exponential 

power  model 

Mean  Square  Error  of  the  exponential  power 

model 

Maximum  Likelihood  estimate  with  discounting 

Mean  Square  Error  of  the  MLEWD 

exponential  regression  estimate 


-  MSEREG   . . .  Mean  Square  Error  of  the  exp. Reg. Estimate 
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APPENDIX  B 
FILES  AND  VARIABLES 


1.  Sample  DISCONT  EXEC  Al  file 

2.  Sample  input  file  :  PT1C5F3  DATA  Al 

3.  Sample  simulation  run,  using  DISCONT  EXEC  Al , 
recorded  session 

4.  Sample  output  GROWTH  EST  Al 

5.  Main  variables  used  for  the  discrete  modification 
of  the  continuous  AMSAA  model  (  AMSAA-D  model  ) 
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1.  Sample  DISCONT  EXEC  Al  file 


&TRACE  OFF 
******************************************************* 

&FN  =  DISCONT 

&FN1  =  GROWTH 

&TYPE  THIS  PROGRAM  USES  A  DISCRETIZED  CONTINUOUS  MODEL 

&TYPE  DO  YOU  NEED  TO  COMPILE  YOUR  PROGRAM  ?  (Y/N) 

&READ  VAR  &R_COMPILE 

&IF  &R_COMPILE  NE  Y  &GOTO  -RUN 

-H  FORTVS  &FN 

&IF  &RC  EQ  0  &GOTO  -RUN 

&TYPE  Your  program  did  not  compile;  check  for  errors. 

&TYPE  DO  YOU  WISH  TO  VIEW  THE  PROGRAM  LISTING  FILE?  (Y/N) 

&READ  VAR  &RSP1 

&IF  &RSP1  EQ  Y  BROWSE  &FN  LISTING  A 

&TYPE  DO  YOU  WISH  TO  XEDIT  THE  PROGRAM  FILE?  (Y/N) 

&READ  VAR  &RESP1 

&IF  &RESP1  NE  Y  &EXIT  1 

&COMMAND  XEDIT  &FN  FORTRAN  A 

&TYPE  DO  YOU  WISH  TO  RUN  THE  PROGRAM  AGAIN?  (Y/N) 

&READ  VAR  &RESP2 

&IF  &RESP2  EQ  Y  &GOTO  -H 

&EXIT  1 

-RUN 
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FILEDEF  10  DISK  PT2C7F3  DATA  Al 

FILEDEF  82  DISK  XAl  NUM  Al 

FILEDEF  84  DISK  XA9  NUM  Al 

FILEDEF  30  DISK  XJRELIAB  LISTING  Al  (LRECL  13  3 

FILEDEF  35  DISK  XPRELIAB  LISTING  Al  (LRECL  133 

FILEDEF  20  DISK  XJTHESIS  OUT  Al 

FILEDEF  25  DISK  XPTHESIS  OUT  Al 

FILEDEF  81  DISK  XPMATRIXA  LISTING  (LRECL  133 

FILEDEF  83  DISK  XJMATRIXA  LISTING  (LRECL  133 

FILEDEF  87  DISK  XPREGMAT  DATA  Al 

FILEDEF  88  DISK  XJREGMAT  DATA  Al 

FILEDEF  90  DISK  XYSTAR  LISTING  (LRECL  133 

FILEDEF  89  DISK  XTRIALS  DATA  Al 

FILEDEF  40  DISK  XEST  OUT  Al 

FILEDEF  50  DISK  XJMLEWD  OUT  Al 

FILEDEF  55  DISK  XPMLEWD  OUT  Al 

FILEDEF  60  DISK  XJMLESP  OUT  Al 

FILEDEF  65  DISK  XPMLESP  OUT  Al 

FILEDEF  7  0  DISK  XJREGEST  OUT  Al 

FILEDEF  75  DISK  XPREGEST  OUT  Al 

FILEDEF  15  DISK  XPWRESl  OUT  Al 

FILEDEF  39  DISK  XPWRES2  OUT  Al 

FILEDEF  49  DISK  XPWRES3  OUT  Al 

FILEDEF  16  DISK  XJV7RES1  OUT  Al 

FILEDEF  38  DISK  XJWRES2  OUT  Al 

FILEDEF  48  DISK  XJWRES3  OUT  Al 
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FILEDEF  52  DISK  XMLEWDl  NUM  Al 
FILEDEF  51  DISK  XMLEWDl  SDV  Al 
FILEDEF  54  DISK  XMLEWD9  NUM  Al 
FILEDEF  53  DISK  XMLEWD9  SDV  Al 
FILEDEF  72  DISK  XREG8  NUM  Al 
FILEDEF  71  DISK  XREG8  SDV  Al 
FILEDEF  74  DISK  XREG16  NUM  Al 
FILEDEF  73  DISK  XREG16  SDV  Al 
FILEDEF  77  DISK  XMIPI  NUM  Al 
FILEDEF  76  DISK  XMIPI  SDV  Al 
FILEDEF  79  DISK  XM2P1  NUM  Al 
FILEDEF  78  DISK  XM2P1  SDV  Al 
FILEDEF  92  DISK  XM3P1  NUM  Al 
FILEDEF  91  DISK  XM3P1  SDV  Al 
FILEDEF  18  DISK  XM1P9  NUM  Al 
FILEDEF  17  DISK  XM1P9  SDV  Al 
FILEDEF  94  DISK  XM2P9  NUM  Al 
FILEDEF  93  DISK  XM2P9  SDV  Al 
FILEDEF  96  DISK  XM3P9  NUM  Al 
FILEDEF  95  DISK  XM3P9  SDV  Al 
FILEDEF  62  DISK  XMLESPl  NUM  Al 
FILEDEF  61  DISK  XMLESPl  SDV  Al 
FILEDEF  64  DISK  XMLESP9  NUM  Al 
FILEDEF  63  DISK  XMLESP9  SDV  Al 
FILEDEF  11  DISK  GROWTH  CONTROL  Al 
FILEDEF  12  DISK  FAILURE  RATE  Al 
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FILEDEF  13  DISK  GROWTH  EST  Al 

FILEDEF  14  DISK  ERROR  EST  Al 

FILEDEF  19  DISK  MSE  COMPARE  Al 

FILEDEF  21  DISK  LLOYD  EST  Al 

FILEDEF  2  2  DISK  FRATE  COMPARE  Al 

FILEDEF  06  TERMINAL 

LOAD  &FN  (START 

&IF  &RC  EQ  0  &SKIP  9 

&TYPE  Your  program  did  not  run  correctly;  check  for 

errors. 

&TYPE  Do  you  wish  to  XEDIT  the  program  file?  (Y) 

&READ  VAR  &RESP3 

&IF  &RESP3  NE  Y  &EXIT  2 

&COMMAND  XEDIT  &FN  FORTRAN  A 

&TYPE  Do  you  wish  to  run  the  program  again?  (Y) 

&READ  VAR  &RESP4 

&IF  &RESP4  EQ  Y  &GOTO  -H 

&EXIT  2 

&TYPE  LLOYD  ESTIMATION/EXPECTATION  OUTPUT  IS  IN  "LLOYD  EST 

Al" 

&TYPE 

&TYPE  CONTROL  OUTPUT  IS  IN  THE  FILE  "GROWTH  CONTROL  Al" 

&TYPE 

&TYPE  YOUR  ESTIMATION  OUTPUT  IS  IN  THE  FILE  &FN1  EST  Al 

&TYPE  DO  YOU  WISH  TO  BROWSE  YOUR  ESTIMATION  OUTPUT?  (Y/N) 

&READ  VAR  &RESP 
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&IF  &RESP  EQ  Y  &COMMAND  BROWSE  &FN1  EST  Al 
&TYPE  PRINT  YOUR  OUTPUT  FILE?   (Y/N) 
&READ  VAR  &RESP7 

&IF  &RESP7  EQ  Y  &COMMAND  PRINT  &FN1  EST  Al 
-REDO 

&TYPE  DO  YOU  WISH  TO  CHANGE  THE  INPUT  DATA  FILE  (Y/N) 
&READ  VAR  &RESP5 

&IF  &RESP5  EQ  Y  XEDIT  &FN  EXEC  Al 

&TYPE  DO  YOU  WISH  TO  RUN  THE  PROGRAM  AGAIN?  (Y/N) 
&READ  VAR  &RESP6 

&RESP56  =  &CONCAT  OF  &RESP5  &RESP6 
&IF  &RESP56  EQ  YY  &GOTO  -H 
&IF  &RESP6  EQ  Y  &GOTO  -RUN 
&EXIT 
2.   Sample  input  file  :  PT1C5F3  DATA  Al 

PT1C5F3         PATTERN  1,  5  FAILURE  CAUSES, 

3    FAILURES/ PHASE 
5  NUMBER  OF  FAILURE  CAUSES 

10  NUMBER  OF  PHASES  (  NPHASE  ) 

1  FIXED  RELIABILITY  OPTION 

(  1:  YES  ;  0:  NO  ) 
3  NUMBER  OF  FAILURES  IN  PHASE  1 

3  NUMBER  OF  FAILURES  IN  PHASE  2 

3  NUMBER  OF  FAILURES  IN  PHASE  3 

3  NUMBER  OF  FAILURES  IN  PHASE  4 

3  NUMBER  OF  FAILURES  IN  PHASE  5 
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3  NUMBER  OF  FAILURES  IN  PHASE  6 

3  NUMBER  OF  FAILURES  IN  PHASE  7 

3  NUMBER  OF  FAILURES  IN  PHASE  8 

3  NUMBER  OF  FAILURES  IN  PHASE  9 

3  NUMBER  OF  FAILURES  IN  PHASE  10 

.85  PROB.  OF  SUCCESS  FROM  CAUSE  1  IN  PHASE  1 

.86  PROB.  OF  SUCCESS  FROM  CAUSE  1  IN  PHASE  2 

.90  PROB.  OF  SUCCESS  FROM  CAUSE  1  IN  PHASE  3 

.91  PROB.  OF  SUCCESS  FROM  CAUSE  1  IN  PHASE  4 

.93  PROB.  OF  SUCCESS  FROM  CAUSE  1  IN  PHASE  5 

.95  PROB.  OF  SUCCESS  FROM  CAUSE  1  IN  PHASE  6 

.97  PROB.  OF  SUCCESS  FROM  CAUSE  1  IN  PHASE  7 

.99  PROB.  OF  SUCCESS  FROM  CAUSE  1  IN  PHASE  8 

.99  PROB.  OF  SUCCESS  FROM  CAUSE  1  IN  PHASE  9 

.998  PROB.  OF  SUCCESS  FROM  CAUSE  1  IN  PHASE  10 

.84  PROB.  OF  SUCCESS  FROM  CAUSE  2  IN  PHASE  1 

.85  PROB.  OF  SUCCESS  FROM  CAUSE  2  IN  PHASE  2 

.87  PROB.  OF  SUCCESS  FROM  CAUSE  2  IN  PHASE  3 

.90  PROB.  OF  SUCCESS  FROM  CAUSE  2  IN  PHASE  4 

.92  PROB.  OF  SUCCESS  FROM  CAUSE  2  IN  PHASE  5 

.95  PROB.  OF  SUCCESS  FROM  CAUSE  2  IN  PHASE  6 

.97  PROB.  OF  SUCCESS  FROM  CAUSE  2  IN  PHASE  7 

.99  PROB.  OF  SUCCESS  FROM  CAUSE  2  IN  PHASE  8 

.99  PROB.  OF  SUCCESS  FROM  CAUSE  2  IN  PHASE  9 

.998  PROB.  OF  SUCCESS  FROM  CAUSE  2  IN  PHASE  10 

•83  PROB.  OF  SUCCESS  FROM  CAUSE  3  IN  PHASE  1 
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84  PROB.  OF  SUCCESS  FROM  CAUSE  3  IN  PHASE  2 

86  PROB.  OF  SUCCESS  FROM  CAUSE  3  IN  PHASE  3 

88  PROB.  OF  SUCCESS  FROM  CAUSE  3  IN  PHASE  4 

90  PROB.  OF  SUCCESS  FROM  CAUSE  3  IN  PHASE  5 

93  PROB.  OF  SUCCESS  FROM  CAUSE  3  IN  PHASE  6 

96  PROB.  OF  SUCCESS  FROM  CAUSE  3  IN  PHASE  7 

98  PROB.  OF  SUCCESS  FROM  CAUSE  3  IN  PHASE  8 

99  PROB.  OF  SUCCESS  FROM  CAUSE  3  IN  PHASE  9 
998  PROB.  OF  SUCCESS  FROM  CAUSE  3  IN  PHASE  10 
8  3  PROB.  OF  SUCCESS  FROM  CAUSE  4  IN  PHASE  1 
84  PROB.  OF  SUCCESS  FROM  CAUSE  4  IN  PHASE  2 
8  5  PROB.  OF  SUCCESS  FROM  CAUSE  4  IN  PHASE  3 

87  PROB.  OF  SUCCESS  FROM  CAUSE  4  IN  PHASE  4 

89  PROB.  OF  SUCCESS  FROM  CAUSE  4  IN  PHASE  5 
92  PROB.  OF  SUCCESS  FROM  CAUSE  4  IN  PHASE  6 

94  PROB.  OF  SUCCESS  FROM  CAUSE  4  IN  PHASE  7 

97  5  PROB.  OF  SUCCESS  FROM  CAUSE  4  IN  PHASE  8 
99  PROB.  OF  SUCCESS  FROM  CAUSE  4  IN  PHASE  9 
998  PROB.  OF  SUCCESS  FROM  CAUSE  4  IN  PHASE  10 
81  PROB.  OF  SUCCESS  FROM  CAUSE  5  IN  PHASE  1 

83  PROB.  OF  SUCCESS  FROM  CAUSE  5  IN  PHASE  2 

84  PROB.  OF  SUCCESS  FROM  CAUSE  5  IN  PHASE  3 
8  6  PROB.  OF  SUCCESS  FROM  CAUSE  5  IN  PHASE  4 
89  PROB.  OF  SUCCESS  FROM  CAUSE  5  IN  PHASE  5 

91  PROB.  OF  SUCCESS  FROM  CAUSE  5  IN  PHASE  6 
94  PROB.  OF  SUCCESS  FROM  CAUSE  5  IN  PHASE  7 
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.961  PROB.  OF  SUCCESS  FROM  CAUSE  5  IN  PHASE  8 

.99  PROB.  OF  SUCCESS  FROM  CAUSE  5  IN  PHASE  9 

.99  8  PROB.  OF  SUCCESS  FROM  CAUSE  5  IN  PHASE  10 

1  NUMBER  OF  TRIALS  AFTER  FAILURE  BEFORE  A 

DISCOUNT  IS  APPLIED 
0.0  FRACTION  EACH  FAILURE  IS  DISCOUNTED 

624712.0       RANDOM  NUMBER  SEED  FOR  GGUBFS  UNIFORM  (0,1) 

GENERATOR 
.75  FRACTION  RELIABILITY  IMPROVES  AFTER  FAILING 

IN  A  PHASE 
500  NUMBER  OF  DESIRED  REPETITIONS  FOR  THE 

SIMULATION 
0  INTERMEDIATE  INPUT  OPTION ( 1 : INT.  OUTPUT; 

0:  NO  INT.  OUTPUT) 
0  SAVE  ALL  MLE  W/  DISCOUNTING  ESTIMATES 

(1:  YES;  0 :  NO  ) 
0  SAVE  ALL  MLE  SINGLE  PHASE  ESTIMATES 

(1:  YES;  0:  NO  ) 
0  SAVE  ALL  UNWT'D  REGRESSION  ESTIMATES 

(1:  YES;  0 :  NO  ) 
0  SAVE  ALL  METHOD  1  WT ' D  REG.  ESTIMATES 

(1:  YES;  0 :  NO  ) 
0  SAVE  ALL  METHOD  2  WT ' D  REG.  ESTIMATES 

(1:  YES;  0:  NO  ) 
0  SAVE  ALL  METHOD  3  WT ' D  REG.  ESTIMATES 

(1:  YES;  0:  NO  ) 
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1  DISCOUNTING  OPTION  (1:  STRAIGHT  %  ;  2:  LLOYD 

METHOD) 
.9  PERCENT  C.I.  FOR  LLOYD  DISCOUNTING  METHOD 

(  MUST  HAVE  A  VALUE  ) 
1  LLOYD  DISCOUNT  INTERVAL 

.03  WEIGHT  FOR  PHASE  1 

.03  WEIGHT  FOR  PHASE  2 

.03  WEIGHT  FOR  PHASE  1 

.03  WEIGHT  FOR  PHASE  2 

.03  WEIGHT  FOR  PHASE  1 

. 15  WEIGHT  FOR  PHASE  2 

. 15  WEIGHT  FOR  PHASE  1 

.15  WEIGHT  FOR  PHASE  8 

.2  WEIGHT  FOR  PHASE  9 

.2  WEIGHT  FOR  PHASE  10 

Sample  simulation  run,  using  DISCONT  EXEC  Al,  recorded 
session 

BEGIN  RECORDING  OF  TERMINAL  SESSION 
Ready;  T=0.01/0.04  08:49:01 
DISCONT 

THIS  PROGRAM  USES  A  DISCRETIZED  CONTINUOUS  MODEL 
DO  YOU  NEED  TO  COMPILE  YOUR  PROGRAM  ?  (Y/N) 
Y 

VS  FORTRAN  COMPILER  ENTERED.   08:49:17 
+2S=$.33 
+2S=$.33 
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**MAIN**  END  OF  COMPILATION  1  ****** 
VS  FORTRAN  COMPILER  EXITED.    08:49:26 

Execution  begins... 

+2S=$.33 

+2S=$.33 

+2S=$.33 

+2S=$.33 

+2S=$.33 

LLOYD  ESTIMATION/EXPECTATION  OUTPUT  IS  IN  "LLOYD  EST  Al" 

CONTROL  OUTPUT  IS  IN  THE  FILE  "GROWTH  CONTROL  Al" 

YOUR  ESTIMATION  OUTPUT  IS  IN  THE  FILE  GROWTH  EST  Al 

DO  YOU  WISH  TO  BROWSE  YOUR  ESTIMATION  OUTPUT?  (Y/N) 

Y 

PRINT  YOUR  OUTPUT  FILE?  (Y/N) 

Y 

DO  YOU  WISH  TO  CHANGE  THE  INPUT  DATA  FILE  (Y/N) 

Y 

DO  YOU  WISH  TO  RUN  THE  PROGRAM  AGAIN?  (Y/N) 

Y 

VS  FORTRAN  COMPILER  ENTERED.   08:50:56 

+2S=$.33 

+2S=$.33 
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**MAIN**  END  OF  COMPILATION  1  ****** 
VS  FORTRAN  COMPILER  EXITED.    08:51:05 


Execution  begins... 

+2S=$.33 

+2S=$.33 

+2S=$.33 

+2S=$. 33 

+2S=$.33 

LLOYD  ESTIMATION/EXPECTATION  OUTPUT  IS  IN  "LLOYD  EST  Al" 

CONTROL  OUTPUT  IS  IN  THE  FILE  "GROWTH  CONTROL  Al" 

YOUR  ESTIMATION  OUTPUT  IS  IN  THE  FILE  GROWTH  EST  Al 

DO  YOU  WISH  TO  BROWSE  YOUR  ESTIMATION  OUTPUT?  (Y/N) 

Y 

PRINT  YOUR  OUTPUT  FILE?  (Y/N) 

N 

DO  YOU  WISH  TO  CHANGE  THE  INPUT  DATA  FILE  (Y/N) 

N 

DO  YOU  WISH  TO  RUN  THE  PROGRAM  AGAIN?  (Y/N) 

N 

Ready;  T=27 . 06/30. 92  08:52:03 

EXEC  REC  OFF 
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4.   Sample  output  GROWTH  EST  Al 


SIMULATION  INPUT  FILE  :  PT1C5F3 


PHASE 

ACTREL 

LINREL 

MSEL 

RELEXP 

HSEE 

MLEWD 

MSEWD 

REGEST 

MSEREG 

1 

.39842 

.39805 

.0397 

.55853 

.0375 

.33786 

.0517 

.36841 

.0553 

2 

.42811 

.40747 

.0404 

.56384 

.0304 

.34822 

.0547 

.37926 

.0569 

3 

.48079 

.45315 

.0418 

.59035 

.0250 

.39846 

.0564 

.43754 

.0432 

4 

.53924 

.51145 

.0423 

.62577 

.0217 

.46259 

.0561 

.50047 

.0374 

5 

.60995 

.56290 

.0437 

.65869 

.0177 

.51918 

.0585 

.56433 

.0296 

6 

.70268 

.65305 

.0298 

.71604 

.0122 

.61836 

.0402 

.64388 

.0213 

7 

.79812 

.75894 

.0221 

.79329 

.0103 

.73483 

.0289 

.73642 

.0147 

8 

.89996 

.88128 

.0071 

.89087 

.0045 

.86940 

.0091 

.84127 

.0069 

9 

.95099 

.93208 

.0043 

.93596 

.0027 

.92529 

.0054 

.90832 

.0030 

10 

.99004 

.98662 

.0002 

.98678 

.0001 

.98528 

.0002 

.96385 

.0009 

Main  variables  used  for  the  discrete  modification  of  the 

continuous  AMSAA  model  (  AMSAA-D  model  ) 

TRTOT        number  of  trials  in  a  phase 

NFAPH(K)     number  of  failures  in  phase  k 

NFAPHK       number  of  failures  in  a  phase 

FRATE       failure  rate  estimate  of  the  AMSAA-D  model 

FRHAT(K)     failure  rate  estimate  for  phase  k  using  the 

AMSAA-D  model 
FSHAT(K)     minimum   variance   unbiased   failure   rate 

estimate  for  phase  k 
XAHAT,XBHAT  regression  estimates  a,  b  in  one  phase  and 

repetition 
Z**         auxiliary  variables 
AREL(K)      actual  reliability  in  phase  k 
RHAT(K)      failure  rate  estimate  at  phase  k 
XI  In  of  the  total  number  of  trials  up  to  phase  k 

YI  In  of  the  failure  rate  estimate  at  phase  k 
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LSSUM(4,K) 

ANUMER 
ADENOM 
ESTRN(K) 

ESTRNE(K) 

RNHAT(K) 

RNHATE(K) 

AVFR(K) 
AVRN(K) 

MSE(K) 

AVRNE(K) 

MSEE(K) 

MLEWD(3,K) 

MSEWD(K) 


summands  of  the  least  square  estimate  a  in 
phase  k 

A 

numerator  to  compute  a 

A 

denominator  to  compute  a 

failure  rate  estimate  in  phase  k  using  the 

linear  model 

failure  rate  estimate  in  phase  k  using  the 

exponential  power  model 

summation  of  the  reliability  estimate  using 

the  linear  model 

summation  of  the  reliability  estimate  using 

the  exponential  power  model. 

average  failure  rate  estimate  for  phase  k 

average  reliability  estimate  for  phase  k, 

using  the  linear  model 

Mean   Square   Error   of   the   linear   model 

reliability  estimation 

average  reliability  estimate  for  phase  k, 

using  the  exponential  power  model 

Mean  Square  Error  of  the  exponential  power 

model  reliability  estimation 

average   reliability   using   the   maximum 

likelihood  estimation  with  discounting 

Mean  Square  Error  of  the  MLEwd 
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REGEST(3,K)  average  reliability  using  the  exponential 

regression  estimate 
MSEREG(K)    Mean  Square  Error  of  the  exp.Regr .Estimate 
MSEERR(I,K)  average  Mean  Square  Error  over  all  phases  k 
for  the  i  methods, 
i  =  1  :  AMSAA-D,  linear  model 
i  =  2  :  AMSAA-D,  exponential  power  model 
i  =  3  :  MLEwd 
i  =  4  :  exp.Regr . Estimate 
AVGMSE(I)    average  Mean  Square  Error  over  all  phases  for 
method  i 
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APPENDIX  C 


FAILURE  RATE  FOR  THE  GEOMETRIC  AND  PASCAL  DISTRIBUTION 


The  failure  rate  for  a  discrete  probability  distribution 
is  defined  as 


_  P[N=n]   „  _  -  -  _  (    r   1    \ 

trt   =  — ,n=l,2,3...  (C.l) 

^    P[N>n]  '      '  '  ^  ' 


In  this  treatment  the  random  variable  N  is  the  number  of 
trials  up  to  and  including  the  r  failure;  that  is,  N  has  a 
Pascal  distribution. 

1.   Case  1  :  The  number  of  failures  equals  1. 

In  this  case  the  number  of  Bernoulli  trials,  N, 
required  to  obtain  the  first  failure,  has  a  geometric 
distribution  with  parameter  p.  Thus  : 


P[N  =  n]  =  pri-l(l-p)  ,   n  =  1,2,...   (  C.2  ) 


It  follows  that 


P[N  >  n]  =  p 


-  T^n 
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Therefore  from  equation  (  C.l  ) , the  failure  rate 


function  fj^  is 


f  =  P""^(l-P) 
pn-1 

or 

f  =  1  -  p  .  (  C.3  ) 

The  parameter  p  of  the  geometric  distribution  represents  the 
reliability  of  the  item  under  test  and  therefore  it  follows 
that 

Rtk  =  1  -  ftk  (  C.4  ) 

This  relation  suggests  the  use  of  equation  (II. 6)  to  convert 
the  estimated  failure  rate  into  a  reliability  value;  i.e. 

^tk  =  1  -  ^tk  (  C.5  ) 

Even  when  the  number  of  failures  in  a  phase  is  r  >  1, 
we  are  still  interested  in  modeling  the  growth  of  the 
probability  of  success  on  a  single  trial.  Consequently 
equation  C.5  provides  the  correct  expression  for  an  estimate 
of  reliability  in  terms  of  the  failure  rate  estimate. 
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2.   Case  2  :  The  number  of  failures  equals  r. 

In  this  case  N  has  a  Pascal  distribution  with 
parameters  p  and  r. 


P[N  =  n]  =  (^Ii)p"  ""(1  -  P)"" 


and 

P[N  >  n]  =  P[N  >  n-1]  =  P[Y  <  r-1] 

where  Y  is  binomial  with  parameters  1-p  and  n-1.  Therefore 


P[N  >  n]  =  ""s   ("T^p"-^-^(l  -  p)^ 
j  =  0    J 


and 

(^:];)p^-^(i-p)^ 

^n  =  ^zi : T      (  c-^  ) 

2   (^:^p^-^-^(l-p)^ 
j=o   J 


It  is  important  to  realize  that  even  though  C.6  is  the 
failure  rate  function  when  testing  until  r  failures  occur,  it 
is  not  this  failure  rate  that  we  are  modeling  in  the  AMSAA-D 
model.  We  are  modeling  the  growth  of  p  only;  i.e.  the  growth 
of  the  probability  of  success  (  reliability  )  on  a  single 
trial. 
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APPENDIX  D 


FIGURES  16  TO  45 
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RELIABILITY  OROWTH  ESTIMATION 
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Figure  16  :  Reliability  growth  estimation  with  the  discrete 
AMSAA  model,  pattern  2 
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RELIABILITY  QROVfTH  ESTIMATION 
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Figure  17  :  Reliability  growth  estimation  with  the  discrete 
AMSAA  model,  pattern  4 


63 


RCUIABIUITY    OROWTH    ESTIMATION 

1 

Pattern    1,    oauaas    ■    3,    f •ilur«/ph«»»    ■    3 

I                                               I                       1                       I                       .                       .                       .                       j                      T                      .                       .                       1                       •                       ■                       ■                        1                       •■                       ■                       1^       J 

j                                                                                                                                 ^^.<r<*i     - 

-              1 

U<^' C-"'             i    ■ 

Ay<::.. !.: 

8.9 

:| 

'i 

/*  ^  '■      \ 

■     : 

y  *  ,"        !                        !    - 

^..s 

k    i 

f^^;-;" i i 

•H 

• 

•  V*  -'   '                   '■                            i    ■ 

-* 

. 

'y.'A/          \             \  - 

i 

- 

.   x^y  ♦                           i  - 

-4 

i 

:                                                i 

i  ••' 

--:: •qti|i«i.„r«.l, | 

"■                  ^y^i^--        ■ 

.+.    n<N)»i-r<N) 

W--' 

i 

....      R<NJ«axp<f  <N)) 

-■..     MLC  ;  with  dlactauntlria              | 

«.e 

i 

■j^jii'^-'-     J*' ;»■ 

■ !     — 

i 

V^ 

<r<^  t         ,-''          ,' 

-x       ■xpjRaoi*»»«tlni«t^    - 

- 

*'         ^.'                        \                                          \                                          \                                          j      . 

- 

X-- 

:        •'                                 \                                             •                                             : 

:    •'                                              '•                                                     '•                                                     1 

" 

•                                         ! 

0.6 

j 

0,- 

1 I i i 1 Z—i — i — 1 — . — 1 — i — 

i 

1 

', i , , 

• 

a                       4                       e                       e                      10 

plt>mwm 

re^fin  squMTE  em7on 

•  .ee 

Pattam   1,    oau«a«    ■    3,    failura/pi^aaa    "    3 

' 

1        1        1         1        1        1        1         1        1         1        >         .         .         1        ■        1         t        1 

^ 1 .i ; i 

!                               !               R(N)»l-:f<N)                      i     ^ 

- 

IT'' 

a  ' 

'          i\            ••» .+.       R(N)=BSp(f  <N)) 

I 

; 

i      s                                                                    .*       MLC   with  dlactxjntlnt 

- 

0.04 

1 

■ \ 1 \. i i  

.      j 

V                                                               ..«  ■     axp.Ra^.Eatiaiata 

.   i 

■\                            '•■                   '■■                             :                                                •• 

- 

- 

1                      ^•.. 

- 

0.03 

1! 

■ 

• — ' 

""        ~          i    ^^""^^.v^                                  V 

.•.-. i i 

i 

:                                     \ 

" 

0.02 

*... 

1 

V            ^v^^            :                                             1 

7  1 

•  ■< 

■ H r**^^. 1 ^. •..■! 

• 

- 

"  t  ^ 

\                             T\                                            ': 

: 

**  —  ^ 

X                           1    \              *.                       :                                                       1 

-       j 

1              *■            i^          *.^           j\^ 

-    i 

0.01 

j                              j            !!!♦..-,> .'.^t.-i            ^V       .."•_....; — 

;  1 

I 

1                         ""^tS^.          i  - 

0 

■                                         i                                         •                                          '                    ▼~'*^tt>'>i  ^M 

~  i 

1 

i       .       .       .       i       .       .       .       i       .       .       .       i       .       •       • 

• 

a                       4                       e                       0                      10 

pt-taaaa 

Figure  18  :  Reliability  growth  estimation  and  Mean  Square 
Error,  pattern  1,  3  failure  causes,  3  failures/phase 
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RCLZABILXTY  QPOUTH  ESTIMATION 
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Figure  19  :  Reliability  growth  estimation  and  Mean  Square 
Error,  pattern  2,  3  failure  causes,  3  failures/phase 
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Figure  20  :  Reliability  growth  estimation  and  Mean  Square 
Error,  pattern  3,  3  failure  causes,  3  failures/phase 
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Figure    21     :     Reliability    growth    estimation     and    Mean    Square 
Error,    pattern    4,    3    failure    causes,    3    failures/phase 
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Figure  22  :  Reliability  growth  estimation  and  Mean  S 
Error,  pattern  5,  3  failure  causes,  3  failures/phase 
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Figure  23  :  Reliability  growth  estimation  and  Mean  Square 
Error,  pattern  6,  3  failure  causes,  3  failures/phase 
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Figure  24  :  Reliability  growth  estimation  and  Mean  Square 
Error,  pattern  1,  5  failure  causes,  1  failure/phase 
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Figure  25  :  Reliability  growth  estimation  and  Mean  Square 
Error,  pattern  2,  5  failure  causes,  1  failure/phase 
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Figure  26  :  Reliability  growth  estimation  and  Mean  Square 
Error,  pattern  3,  5  failure  causes,  1  failure/phase 
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Figure  27  :  Reliability  growth  estimation  and  Mean  Square 
Error,  pattern  4,  5  failure  causes,  1  failure/phase 


73 


i.ea 


e.es 


a    c.es 


e.43 


•  .S3  - 


•  .16  - 


•  .  13 


•  ••e 


•  .•e  - 


i.e3  - 


RCLZABZLZTY  OmjWTH  BSTIMATION 

Pattam  B.  e«ua««  ■  B.  fallur«/phia««  >  1 

'  ■'■■■'■■I 


"T 


t 

.--♦•■ 


,  _  ♦ 


actual  iral. 
R<N)»l-ir(N) 
R<N>'aMp<r<N>> 
riUE  with  diaooun 
■xp.  Ra(j.  Catimat 


ting 


!• 


pHaaaa 


MEAN  squARe  ERnon 

Pattam  6»  eauaaa  »  g^  fallura/ptTaaa  ■  1 


! 

1     1     1     1     1     1 

i 

! 

1            1            1            1            1            1            1            r           1            1           1            1            1            1 

1 :*:;■:  "PCN)iii-?fCH) |-~ 

-     : 

X,                     1 

i        ■        R(N>>a)<p(r(N)>                1     - 

-     i 

•B        MLE   with   diacountlng     ■ 

■■■|  ■ 

i' •; \ 

i 

/ 

to  - 

• 

■    ■     ■      T           S    ■•        - 

- ;.'  _ , '** ' \ • ' 

•                     >       ♦  _ 

i 

*                                * 

'^~*-.   i    ^,.^----.--"*~-~4  - 

', 

^  t                                                          i    " 

\ 

i 

■>, 

i 

: 

* •■- 

\-* • 1   • + • i 

■■-■^-,.      i                                        I  " 

i 

__..i 

i                     i     '""■•« i ■« * 

i  __   .      :     : :.. 

i     ...     I     ...     i     ...     i     ...     i     ...     i  "1 

1* 


phaaaa 


Figure  28  :  Reliability  growth  estimation  and  Mean  Square 
Error,  pattern  5,  5  failure  causes,  1  failure/phase 
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Figure  29  :  Reliability  growth  estimation  and  Mean  Square 
Error,  pattern  6,  5  failure  causes,  1  failure/phase 
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Figure  30  :  Reliability  growth  estimation  and  Mean  Square 
Error,  pattern  1,  7  failure  causes,  3  failures/phase 
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Figure  31  :  Reliability  growth  estimation  and  Mean  Square 
Error,  pattern  2,  7  failure  causes,  3  failures/phase 
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Figure    32     :     Reliability    growth    estimation    and    Mean    Square 
Error,  pattern  3,  7  failure  causes,  3  failures/phase 
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Figure  33  :  Reliability  growth  estimation  and  Mean  Square 
Error,  pattern  4,  7  failure  causes,  3  failures/phase 
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Figure  34  :  Reliability  growth  estimation  and  Mean  Square 
Error,  pattern  5,  7  failure  causes,  3  failures/phase 
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Figure  35  :  Reliability  growth  estimation  and  Mean  Square 
Error,  pattern  6,  7  failure  causes,  3  failures/phase 
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Figure  36  :  Mean  Square  Error  for  both  AMSAA-D  models  as  a 
function  of  phases  and  number  of  failure  causes,  pattern  2, 
failures/phase  =  3 
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Figure  37  :  Mean  Square  Error  for  both  AMSAA-D  models  as  a 
function  of  phases  and  number  of  failure  causes,  pattern  3, 
failures/phase  =  3 
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Figure  38  :  Mean  Square  Error  for  both  AMSAA-D  models  as  a 
function  of  phases  and  number  of  failure  causes,  pattern  4, 
failures/phase  =  3 
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Figure  39  :  Mean  Square  Error  for  both  AMSAA-D  models  as  a 
function  of  phases  and  number  of  failure  causes,  pattern  5, 
failures/phase  =  3 
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Figure  40  :  Mean  Square  Error  for  both  AMSAA-D  models  as  a 
function  of  phases  and  number  of  failure  causes,  pattern  6, 
failures/phase  =  3 
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Figure  41  :  Mean  Square  Error  for  both  AMSAA-D  models  as  a 
function  of  phases  and  failures/phase,  pattern  2,  failure 
causes  =  5 
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Figure  42  :  Mean  Square  Error  for  both  AMSAA-D  models  as  a 
function  of  phases  and  failures/phase,  pattern  3,  failure 
causes  =  5 
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Figure  43  :  Mean  Square  Error  for  both  AMSAA-D  models  as  a 
function  of  phases  and  failures/phase,  pattern  4,  failure 
causes  =  5 
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Figure  44  :  Mean  Square  Error  for  both  AMSAA-D  models  as  a 
function  of  phases  and  failures/phase,  pattern  5,  failure 
causes  =  5 
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Figure  45  :  Mean  Square  Error  for  both  AMSAA-D  models  as  a 
function  of  phases  and  failures/phase,  pattern  6,  failure 
causes  =  5 
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APPENDIX  E 


GLOSSARY  OF  THE  DIFFERENT  RELIABILITY  GROWTH  MODELS 

1.  Maximum  Likelihood  Estimate  Model 

2.  The  Exponential  Regression  Model 

3 .  The  Discrete  AMSAA  Model 

4 .  The  AMSAA-D  Model 

a.  Linear  AMSAA-D  Model 

b.  Exponential  Power  AMSAA-D  Model 
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1.   Maximum  Likelihood  Estimate  Model 

A 

The  reliability  estimate  R.  ,  of  the  true  reliability  R  is 

computed  as 

^      total  trials  -  total  failures 

^tk  "         total  trials 


If  failure  discounting  is  applied  then  the  reliability 

A 

estimate  R.  ,  is  evaluated  as 

^      total  trials  -  total  adjusted  failures 
^tk  "  total  trials 


The  estimation  is  performed  after  each  phase  and,  if 
failure  discounting  is  applied,  the  actual  test  data  are 
updated  with  the  use  of  test  data  from  previous  phases.  This 
model  is  denoted  as  MLEwd. 
2.   The  Exponential  Regression  Model 

The  exponential  regression  model  obtains  sequentially 

A 

updated  estimates  R.,  of  the  true  reliability  R.  ,  after  the 
k   phase.  The  basic  model  for  R.,  is  : 


J^tk  =  1  -  ^'''^ 


The  parameter  A,  =  a  +  ^k  is  estimated  by  A,  where 


'^k  =  "k  +  ^k^ 
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The  estimates  a,  and  ^,  are  least  square  regression  estimates 

A  A 

for  a     and  ^.  Equations  for  a,   and  ^,   are  developed  in 
Reference  4. 

A 

The   exponential   regression   estimate   R.  ,   of   the   true 
reliability  R.  ,  is  then  finally  : 


S^,^  =  1  -  e-^^k  -^  V)  ,    k  =  1,2, 


This  model  is  denoted  as  exp.Regr .Estimate. 
3 .   The  Discrete  AMSAA  Model 

In  the   Discrete  AMSAA  model,   the   equation   for  the 
probability  of  success,  R,  ,  on  each  trial  in  the  i    testing 


phase  is  given  by 


where 


(1  -  R,^)N^  =  XT^  -  XT^_^ 


N,   :  number  of  tests  in  test  phase  i 


T,   =   2   N. 


After  each  new  test  phase  is  completed  the  maximum  likelihood 
estimates  for  X  and  ^  are  recomputed  to  provide  the  MLE  for 
R,  .  The  two  equations  which  must  be  solved  after  testing  is 
completed  in  phase  k  are  as  follows  [Ref.  6]: 
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^  I  lATf  -  ATf_j     N.-  -  ATf  +  ATf.^ 


N.  -  M, 

^0 


ATf  In  T,  -  ATf_ilnT,_i 


and, 


M.- 


N.  -  M. 


y  .  1    r  1 

^  I  [ATf  -  ATf_i]   [n.-  -  ATf  +  ATf_i 


Numerical  methods  and  a  computer  are  needed  to  solve  these 

equations. 

4.   The  AMSAA-D  Model 

The  Continuous  AMSAA  Model  expresses  the  instantaneous 
failure  rate  f__   after  TT,  time  units  of  testing  by 


f^^   =  b(l  -  a)TTj^ 


-a 


where  TT,  is  the  total  accumulated  test  time  over  k  phases. 
This  model  is  also  known  as  the  Duane  Model.  After  examining 
a  large  variety  of  data,  Duane  noticed  that  the  plot  of  the 
cumulative  failure  rate  \^  versus  cumulative  test  time  on  log- 
log  paper  produced  a  straight  line.  From  this  he  inferred  a 
linear  relationship  between  log  A„  and  log  T.  Consequently, 
he  wrote  the  following  model  for  cumulative  failure  rate 

A^  =  kT"^ 
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V,         \     N(T) 
where    A_  =  — ^ — - 
1     T 


Thus 


N(T)   :  total  failures  up  to  time  T 
k  and  a  are  unknown  parameters. 


N(T)  =  kT-"-"^ 


The  instantaneous  failure  rate  f„  is  the  change  in  N(T)  per 


unit  time.  Thus 


f^  =  ^^  =  (1  -  a)kT-^ 


"^'  The  AMSAA  model  departs  at  this  point  from  Duane  by  using 
maximum  likelihood  methods  to  estimate  the  unknown  parameters. 
Letting  a  =  1-^  we  have 

f^  =  ^kT^~^ 
which  is  the  Weibull  failure  rate  function.  Consequently  AMSAA 
develops  maximum  likelihood  estimates  for  ^  and  k  assuming 
time  to  failure  within  a  phase  has  a  Weibull  distribution. 
This  assumption  makes  the  accuracy  of  their  model  succeptible 
to  variations  in  the  underlying  failure  distribution.  Woods 
[Ref.  4]  uses  regression  methods  to  estimate  the  parameters 
and  does  not  assume  an  underlying  distribution.  This  makes  his 
procedure  more  robust.  Similar  procedures  are  followed  in  the 
AMSAA-D  model. 
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The  AMSAA-D  model  relates  the  failure  rate  f^jr  and  the  end 
of  k  test  phases  to  the  total  number  of  failures  N^j^  across 
the  first  k  test  phases  by 


ftk  =  b(l  -  a)Ntk 


a.   Linear  AMSAA-D  Model 

A 

In  this  model  the  reliability  estimate  R.  ,  of  the  true 
reliability  R.,  at  the  end  of  phase  k  is  computed  as 


^tk  "  ^  ■  ^tk* 


b.   The  exponential  Power  AMSAA-D  Model 

A 

In  this  model  the  reliability  estimate  R.  ,  of  the  true 
reliability  R.  ,  at  the  end  of  phase  k  is  computed  as 

A 

^tk  "  ^ 
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